Statistical Skier

With the first period of WC racing in the books and the next one just around the corner, I’m going to look back at the results of the North American skiers so far this season.  The theme of this post is context and expectations.

One of the reasons that I started this blog was that I felt as though the tools we use to measure performance in skiing don’t always match up with our expectations and this can lead to a lot of needless frustration.  The general paradigm for measuring performance in XC skiing is to examine only an athlete’s best few races over some time period.  FIS averages your best five races over the previous calendar year for their point lists; EISA picks out each skiers best two classic and freestyle races (if I recall correctly), and so on.

We may not be explicitly aware of it, but this methodology stems from an important observation: once in a while good skiers can have truly terrible races.  Averaging is particularly sensitive to extremely small or large values, so one terrible FIS point race could have a huge effect if we were to look at all of someones results.  This is an excellent observation, but I don’t think people are necessarily aware of the consequences.

Performance measures that look only at an athlete’s best few races are a terrible indicator of how well they typically ski.  Instead, it’s measuring how well they have skied at their very best.  It’s an estimate of an upper limit, not a “typical value”.

This, combined with short memories, leads to some strange tendencies, I think.  When we think about how good a skier is, we tend to think about their best results: Kris Freeman’s 4th place finishes, Kikkan Randall’s sprint podiums, etc.  But those aren’t necessarily indicative of how well they ski from week to week.

With all that in mind, let’s look at some graphs.

Note the slightly unusual scale on the y axis.  I’ve been tinkering with this for a while now, and I’ve grown to like using percent back from the median skier.  Negative values are good (ahead of the median skier) positive values are bad (behind the median skier).  It turns out that it mostly (but not completely!) eliminates discrepancies between mass start and interval start races, it’s easy to interpret, and there are some technical mathematical reasons why I like it as well.

The red dots are this season’s results.  I’ve highlighted last year’s Olympic results in blue, which I’ll return to in a moment.  I will also point out the one instance here where percent back from the median doesn’t capture everything as we’d like: Devon Kershaw’s 5th place in the Olympic 50k last year.  His percent back from the median skier isn’t all that low, for a 5th place finish.  This suggests that (a) the pace might have been fairly slow and (b) as with all 50k’s, the field was likely somewhat reduced.

There are a bunch of skiers here who either don’t typically do distance races (Simi Hamilton, Andrew Newell, Stefan Kuhn) or who we just don’t have a ton of data on (Noah Hoffman).

Kris Freeman has had a fairly solid start to the season, in my reading of the data.  On par with his non-Olympic races last year, and definitely better than two years ago.  And that’s including two races that he reportedly didn’t consider being very good: one due to a lingering illness, and one just sub-par day.

The Canadian men have come in for some criticism, but I just don’t see it.  Babikov seems a little off his game and George Grey has had results at the low end of what he managed last year, but Kershaw and Harvey have been skiing just fine, I think.  This is a good example, I think, of how our expectations can become skewed.  Each of these guys managed to have one of their best results of the season at the Olympics last year, but they didn’t ski that fast every day.  There’s a danger of only remembering those outstanding races and forgetting all the days when these guys were just having “good” days in 15th, 20th, or even worse.

What if you don’t like my percent back from the median skier measure?  Consider this.  Over the past two seasons, 50% of Devon Kershaw’s major international results have been between 10th and 37th.  For Alex Harvey, the range is 17th to 34th and for Babikov 12th and 38th.

So far this season, Kershaw has managed the following: 9th, 9th, 13th, 22nd and 58th.  For Harvey, it’s 17th, 24th, 34th and 38th.  Compared to each skier’s best recent results (<10th or <17th), these don’t look so good.  But compared to how they’ve typically skied, it’s right on target.  It’s one thing to expect people to improve year to year; it’s entirely another thing to pick out their best 5 races from one season and then expect that to become “normal” the next year.

Now, for Babikov, 50% of his results over the last two years have been between 12th and 38th, and this season he’s seen a 16th, 31st, 38th, 54th and 83rd.  Not as good as Harvey or Kershaw, but not a disaster.  Three races within his normal range, albeit two of those at the lower end, one good race and two terrible ones.  George Grey has been a little more unambiguously behind what’s typical for him, which would be between 17th and 44th.  This year he’s managed a 32nd, 78th and 79th.  That’s certainly less impressive.

But the bottom line here is that we shouldn’t pick out a handful of extraordinary races from last season and automatically expect that to become normal.

There are fewer North American women doing distance events:

Things would have been looking really bad for Morgan Arritola were it not for last weekend’s race in La Clusaz.  But again, I think it’s a mistake to get too worked up over this one good weekend.  We know that Arritola and Stephen can ski that fast from time to time, but it’s not reasonable to expect them to do week in and week out until they actually string several of these together in a row.

Kikkan Randall has had some very strong (for her) distance results so far.  Last season she was mostly right around the median skier, or a bit below.  So far this year, she’s rattled off three races that are better than nearly all her efforts last season.  I suspect, though, that if her sprinting continues as it has been, we won’t see her in distance events much more outside the Tour de Ski.

The Canadian women have had a few decent results (for them) but they’re also probably more focused on sprinting at the moment.

Another way to look at how people are doing is to compare their results to how they’ve done over the equivalent time period in other seasons:

So this only includes pre-Christmas races from each season.  With the exception of last weekend’s mass start race, Babikov looks a bit behind where he’s been at this point in the season before, as does George Grey.  Things look about the same for Freeman, but both Harvey and Kershaw seem to be skiing better than they have early in the season.

The women’s picture is largely unchanged:

A similar look at the sprinters will be up later.

[ad#AdSenseBanner]